Gaussian Statistics and Unsupervised Learning 1

This tutorial presents the properties of the Gaussian probability density function. Subsequently, supervised and unsupervised pattern recognition methods are treated. Supervised classification algorithms are based on a labeled data set. The knowledge about the class membership of this training data set is used for the classification of new samples. Unsupervised learning methods establish clusters from an unlabeled training data set. Clustering algorithms such as the K-means, the EM (expectation-maximization) algorithm, and the Viterbi-EM algorithm are presented. Usage To make full use of this tutorial you have to 1. download the file Gaussian.zip which contains this tutorial and the accompanying Matlab programs. 2. Unzip Gaussian.zip which will generate a subdirectory named Gaussian/matlab where you can find all the Matlab programs. 3. Add the path Gaussian/matlab to the matlab search path, for example with a command like addpath(’C:\Work\Gaussian\matlab’) if you are using aWindows machine, or by using a command like addpath(’/home/jack/Gaussian/matlab’) if you are on a Unix/Linux machine. Sources This tutorial is based on • EPFL lab notes “Introduction to Gaussian Statistics and Statistical Pattern Recognition” by Hervé Bourlard, Sacha Krstulović, and Mathew Magimai-Doss. 1 Gaussian statistics 1.1 Samples from a Gaussian density Useful formulas and definitions: • The Gaussian probability density function (pdf) for the d-dimensional random variable x N (μ,Σ) (i.e., variable x ∈ R following the Gaussian, or Normal, probability law) is given by: g(μ,Σ)(x) = 1 √ 2π d√ det (Σ) e− 1 2 (x−μ) TΣ−1(x−μ) (1) where μ is the mean vector and Σ is the covariance matrix. μ and Σ are the parameters of the Gaussian distribution. • The mean vector μ contains the mean values of each dimension, μi = E(xi), with E(x) being the expected value of x. 1http://www.igi.tugraz.at/lehre/CI/tutorials/Gaussian.zip